Initializing TensorFlow...
Running train.train_progressive_gan()...
Streaming data using dataset.TFRecordDataset...
Dataset shape = [1, 64, 64]
Dynamic range = [0, 255]
Label size    = 0
Constructing networks...

G                           Params      OutputShape             WeightShape             
---                         ---         ---                     ---                     
latents_in                  -           (?, 128)                -                       
labels_in                   -           (?, 0)                  -                       
lod                         -           ()                      -                       
4x4/Dense                   262272      (?, 128, 4, 4)          (128, 2048)             
4x4/Conv                    147584      (?, 128, 4, 4)          (3, 3, 128, 128)        
ToRGB_lod4                  129         (?, 1, 4, 4)            (1, 1, 128, 1)          
8x8/Conv0_up                147584      (?, 128, 8, 8)          (3, 3, 128, 128)        
8x8/Conv1                   147584      (?, 128, 8, 8)          (3, 3, 128, 128)        
ToRGB_lod3                  129         (?, 1, 8, 8)            (1, 1, 128, 1)          
Upscale2D                   -           (?, 1, 8, 8)            -                       
Grow_lod3                   -           (?, 1, 8, 8)            -                       
16x16/Conv0_up              147584      (?, 128, 16, 16)        (3, 3, 128, 128)        
16x16/Conv1                 147584      (?, 128, 16, 16)        (3, 3, 128, 128)        
ToRGB_lod2                  129         (?, 1, 16, 16)          (1, 1, 128, 1)          
Upscale2D_1                 -           (?, 1, 16, 16)          -                       
Grow_lod2                   -           (?, 1, 16, 16)          -                       
32x32/Conv0_up              147584      (?, 128, 32, 32)        (3, 3, 128, 128)        
32x32/Conv1                 147584      (?, 128, 32, 32)        (3, 3, 128, 128)        
ToRGB_lod1                  129         (?, 1, 32, 32)          (1, 1, 128, 1)          
Upscale2D_2                 -           (?, 1, 32, 32)          -                       
Grow_lod1                   -           (?, 1, 32, 32)          -                       
64x64/Conv0_up              73792       (?, 64, 64, 64)         (3, 3, 64, 128)         
64x64/Conv1                 36928       (?, 64, 64, 64)         (3, 3, 64, 64)          
ToRGB_lod0                  65          (?, 1, 64, 64)          (1, 1, 64, 1)           
Upscale2D_3                 -           (?, 1, 64, 64)          -                       
Grow_lod0                   -           (?, 1, 64, 64)          -                       
images_out                  -           (?, 1, 64, 64)          -                       
---                         ---         ---                     ---                     
Total                       1406661                                                     


D                           Params      OutputShape             WeightShape             
---                         ---         ---                     ---                     
images_in                   -           (?, 1, 64, 64)          -                       
lod                         -           ()                      -                       
FromRGB_lod0                128         (?, 64, 64, 64)         (1, 1, 1, 64)           
64x64/Conv0                 36928       (?, 64, 64, 64)         (3, 3, 64, 64)          
64x64/Conv1_down            73856       (?, 128, 32, 32)        (3, 3, 64, 128)         
Downscale2D                 -           (?, 1, 32, 32)          -                       
FromRGB_lod1                256         (?, 128, 32, 32)        (1, 1, 1, 128)          
Grow_lod0                   -           (?, 128, 32, 32)        -                       
32x32/Conv0                 147584      (?, 128, 32, 32)        (3, 3, 128, 128)        
32x32/Conv1_down            147584      (?, 128, 16, 16)        (3, 3, 128, 128)        
Downscale2D_1               -           (?, 1, 16, 16)          -                       
FromRGB_lod2                256         (?, 128, 16, 16)        (1, 1, 1, 128)          
Grow_lod1                   -           (?, 128, 16, 16)        -                       
16x16/Conv0                 147584      (?, 128, 16, 16)        (3, 3, 128, 128)        
16x16/Conv1_down            147584      (?, 128, 8, 8)          (3, 3, 128, 128)        
Downscale2D_2               -           (?, 1, 8, 8)            -                       
FromRGB_lod3                256         (?, 128, 8, 8)          (1, 1, 1, 128)          
Grow_lod2                   -           (?, 128, 8, 8)          -                       
8x8/Conv0                   147584      (?, 128, 8, 8)          (3, 3, 128, 128)        
8x8/Conv1_down              147584      (?, 128, 4, 4)          (3, 3, 128, 128)        
Downscale2D_3               -           (?, 1, 4, 4)            -                       
FromRGB_lod4                256         (?, 128, 4, 4)          (1, 1, 1, 128)          
Grow_lod3                   -           (?, 128, 4, 4)          -                       
4x4/MinibatchStddev         -           (?, 1, 4, 4)            -                       
4x4/Conv                    148736      (?, 128, 4, 4)          (3, 3, 129, 128)        
4x4/Dense0                  262272      (?, 128)                (2048, 128)             
4x4/Dense1                  129         (?, 1)                  (128, 1)                
scores_out                  -           (?, 1)                  -                       
labels_out                  -           (?, 0)                  -                       
---                         ---         ---                     ---                     
Total                       1408577                                                     

Building TensorFlow graph...
Setting up snapshot image grid...
Setting up result dir...
Saving results to /scratch/users/suihong/ProGAN_MultiChannel_Reusults_ConditionedtoMultiConditions_TF/002-Pro-GAN-Unconditional-related/114-pgan-unconditional-2gpu
Training...
tick 1     kimg 640.0    lod 4.00  minibatch 32   time 16m 13s      sec/tick 972.6   sec/kimg 1.52    maintenance 86.4
tick 2     kimg 1280.0   lod 3.00  minibatch 32   time 34m 52s      sec/tick 1105.2  sec/kimg 1.73    maintenance 14.2
tick 3     kimg 1920.0   lod 3.00  minibatch 32   time 51m 37s      sec/tick 1004.6  sec/kimg 1.57    maintenance 0.3
tick 4     kimg 2560.0   lod 2.00  minibatch 32   time 1h 12m 10s   sec/tick 1232.6  sec/kimg 1.93    maintenance 0.3
tick 5     kimg 3200.0   lod 2.00  minibatch 32   time 1h 31m 33s   sec/tick 1162.6  sec/kimg 1.82    maintenance 0.2
tick 6     kimg 3840.0   lod 1.00  minibatch 32   time 1h 54m 31s   sec/tick 1378.0  sec/kimg 2.15    maintenance 0.1
tick 7     kimg 4480.0   lod 1.00  minibatch 32   time 2h 16m 28s   sec/tick 1317.4  sec/kimg 2.06    maintenance 0.3
tick 8     kimg 5120.0   lod 0.00  minibatch 32   time 2h 46m 03s   sec/tick 1773.9  sec/kimg 2.77    maintenance 0.5
tick 9     kimg 5760.0   lod 0.00  minibatch 32   time 3h 14m 19s   sec/tick 1696.0  sec/kimg 2.65    maintenance 0.3
tick 10    kimg 6400.0   lod 0.00  minibatch 32   time 3h 42m 25s   sec/tick 1685.7  sec/kimg 2.63    maintenance 0.2
tick 11    kimg 7040.0   lod 0.00  minibatch 32   time 4h 10m 40s   sec/tick 1692.8  sec/kimg 2.64    maintenance 2.5
tick 12    kimg 7680.0   lod 0.00  minibatch 32   time 4h 38m 57s   sec/tick 1696.3  sec/kimg 2.65    maintenance 0.3
tick 13    kimg 8320.0   lod 0.00  minibatch 32   time 5h 07m 18s   sec/tick 1700.8  sec/kimg 2.66    maintenance 0.2
tick 14    kimg 8960.0   lod 0.00  minibatch 32   time 5h 35m 32s   sec/tick 1693.8  sec/kimg 2.65    maintenance 0.2